Model Selection for Topic Models via Spectral Decomposition

Correctly choosing the number of topics plays an important role in successfully applying topic models to real world applications. Following the latest tensor decomposition framework by Anandkumar et al., we make the first attempt to provide theoretical analysis on the number of topics under Latent Dirichlet Allocation model. With mild conditions, our method provides accessible information on the number of topics, which includes both upper and lower bounds. Experimental results on synthetic datasets demonstrate that our proposed bounds are correct and tight. Furthermore, using Gaussian Mixture Model as an example, we show that our methodology can be easily generalized for analyzing the number of mixture components in other mixture models.